1. Field of the Invention
This invention relates to a method of and a system for processing a signal, and more particularly to a method of and a system for processing a signal in which deterioration in high frequency components of a signal, which gives rise to a problem especially in a digital signal processing involving linear interpolation, is corrected.
2. Description of the Related Art
When carrying out frame rate conversion of an animation image, rotation of an image, or enlargement/contraction of an image is carried out on a.sampled image signal or when pitch conversion is carried out on a sampled sound signal, signal components for points other than sampling points (interpolating points) are required. As a method of obtaining signal components for points other than sampling points, there have been known methods in which linear interpolation is employed as disclosed, for instance, in Japanese Unexamined Patent Publication No. 2(1990)-294784 and Journal of Japanese Academy of Printing (vol.32, No.5, 1995, xe2x80x9cIntroduction to Prepressxe2x80x9d).
However, it has been known that linear interpolation generates low-pass characteristics which depend upon the interpolating positions. Analysis of low-pass characteristics generated by linear interpolation will be described on interpolation of a one-dimensional signal by way of example.
A one-dimensional signal shown in FIGS. 5A and 5B whose frequency band is limited to xe2x88x92xcfx80/T less than xcfx89 less than xcfx80/T will be discussed, hereinbelow. In FIG. 5A, f(t) is the model one-dimensional signal and F(xcfx89) shown in FIG. 5B represents frequency components of the one-dimensional signal f(t). Rendering the one-dimensional signal f(t) discrete at sampling cycles T is equivalent to multiplying the one-dimensional signal f(t) by an impulse train such as represented by the following formula (1).                               s          ⁡                      (            t            )                          =                                            ∑              ∞                                      k              =                              -                ∞                                              ⁢                      δ            ⁡                          (                              t                -                kT                            )                                                          (        1        )            
When thus rendered discrete, the one-dimensional signal f(t) is represented by the following formula (2).                               fs          ⁡                      (            t            )                          =                                            f              ⁡                              (                t                )                                      ·                          s              ⁡                              (                t                )                                              =                                                    ∑                ∞                                            k                =                                  -                  ∞                                                      ⁢                                          f                ⁡                                  (                  t                  )                                            ·                              δ                ⁡                                  (                                      t                    -                    kT                                    )                                                                                        (        2        )            
Frequency components of the signal obtained by rendering discrete the original one-dimensional signal f(t) are obtained by Fourier transformation of fs(t) by the formula (2) and are represented by the following formula (3)                               Fs          ⁡                      (            ω            )                          =                                            υ              ∼                        ⁢                          {                                                f                  ⁡                                      (                    t                    )                                                  ·                                  s                  ⁡                                      (                    t                    )                                                              }                                =                                    1              T                        ⁢                                                            ∑                  ∞                                                  k                  =                                      -                    ∞                                                              ⁢                              F                ⁡                                  (                                      ω                    -                                          k                      ⁢                                                                        2                          ⁢                          π                                                T                                                                              )                                                                                        (        3        )            
As can be understood from formula (3), the spectrum of the signal obtained by rendering discrete the continuous signal f(t) is infinite repetition at cycles of 2xcfx80/T of the spectrum of the original signal f(t) as shown in FIGS. 6A and 6B. Accordingly, it will be understood that, by extracting the frequency components in the range of xe2x88x92xcfx80/T less than xcfx89 less than xcfx80/T by an ideal low-pass filter, a signal Fsb(xcfx89) represented by the following formula (4) is obtained, and the original continuous signal f(t) can be completely restored from the signal which has been rendered discrete by inverse Fourier transformation of the signal Fsb(xcfx89). The ideal low-pass filter is defined to be a filter which outputs signal components in a frequency band of |f| xcfx80/T and cuts signal components in a frequency band of |f| greater than xcfx80/T. The characteristics of the ideal low-pass filter are as shown in FIG. 7.                               Fsb          ⁡                      (            ω            )                          =                              1            T                    ⁢                      F            ⁡                          (              ω              )                                                          (        4        )            
Influence of linear interpolation on the frequency characteristics of a signal will be discussed, hereinbelow. In the following, a discrete signal obtained when the sampling timings of the original signal are retarded by xcfx84xc2x7T are approximated by linear interpolation of fs(t). Then by carrying out frequency analysis on the interpolation signal obtained by the linear interpolation, influence of linear interpolation on the frequency characteristics of a signal will be studied. The interpolation signal is as shown in FIG. 8 and represented by fsxe2x80x2(t) in the following formula (5).                                           fs            xe2x80x2                    ⁡                      (            t            )                          =                              [                                                            (                                      1                    -                    τ                                    )                                ⁢                                  f                  ⁡                                      (                                          t                      -                                              τ                        ⁢                                                  xe2x80x83                                                ⁢                        T                                                              )                                                              +                              τ                ⁢                                  xe2x80x83                                ⁢                f                ⁢                                  {                                      t                    +                                                                  (                                                  1                          -                          τ                                                )                                            ⁢                      T                                                        }                                                      ]                    ·                                                    ∑                ∞                                            k                =                                  -                  ∞                                                      ⁢                          δ              ⁢                              {                                  t                  -                                                            (                                              k                        +                        τ                                            )                                        ⁢                    T                                                  }                                                                        (        5        )            
Fourier transformation of the formula (5) gives the following formula (6).                                                                                           Fs                  xe2x80x2                                ⁡                                  (                  ω                  )                                            =                              xe2x80x83                            ⁢                                                1                  T                                ⁢                                  {                                                                                    (                                                  1                          -                          τ                                                )                                            ⁢                                                                        F                          ⁡                                                      (                            ω                            )                                                                          ·                                                  ⅇ                                                                                    -                              j                                                        ⁢                                                          xe2x80x83                                                        ⁢                            τ                            ⁢                                                          xe2x80x83                                                        ⁢                            T                            ⁢                                                          xe2x80x83                                                        ⁢                            ω                                                                                                                +                                          τ                      ⁢                                              xe2x80x83                                            ⁢                                              F                        ⁡                                                  (                          ω                          )                                                                    ⁢                                              ⅇ                                                  j                          ⁢                                                      xe2x80x83                                                    ⁢                                                      (                                                          1                              -                              τ                                                        )                                                    ⁢                          T                          ⁢                                                      xe2x80x83                                                    ⁢                          ω                                                                                                      }                                *                                                                                                        xe2x80x83                            ⁢                                                                    ∑                    ∞                                                        k                    =                                          -                      ∞                                                                      ⁢                                                      δ                    ⁡                                          (                                              ω                        -                                                  k                          ⁢                                                                                    2                              ⁢                              π                                                        T                                                                                              )                                                        ·                                      ⅇ                                                                  -                        j                                            ⁢                                              xe2x80x83                                            ⁢                      τ                      ⁢                                              xe2x80x83                                            ⁢                      T                      ⁢                                              xe2x80x83                                            ⁢                      ω                                                                                                                                              =                              xe2x80x83                            ⁢                                                                    1                    T                                    ⁡                                      [                                                                  {                                                                              (                                                          1                              -                              τ                                                        )                                                    +                                                      τⅇ                                                          j                              ⁢                                                              xe2x80x83                                                            ⁢                              T                              ⁢                                                              xe2x80x83                                                            ⁢                              ω                                                                                                      }                                            ·                                              F                        ⁡                                                  (                          ω                          )                                                                    ·                                              ⅇ                                                                              -                            j                                                    ⁢                                                      xe2x80x83                                                    ⁢                          τ                          ⁢                                                      xe2x80x83                                                    ⁢                          T                          ⁢                                                      xe2x80x83                                                    ⁢                          ω                                                                                      ]                                                  *                                                                                                        xe2x80x83                            ⁢                              [                                                      ⅇ                                                                  -                        j                                            ⁢                                              xe2x80x83                                            ⁢                      τ                      ⁢                                              xe2x80x83                                            ⁢                      T                      ⁢                                              xe2x80x83                                            ⁢                      ω                                                        ·                                                                                    ∑                        ∞                                                                    k                        =                                                  -                          ∞                                                                                      ⁢                                          δ                      ⁡                                              (                                                  ω                          -                                                      k                            ⁢                                                                                          2                                ⁢                                π                                                            T                                                                                                      )                                                                                            ]                                                                                        =                              xe2x80x83                            ⁢                                                1                  T                                ⁢                                                                            ∑                      ∞                                                              k                      =                                              -                        ∞                                                                              ⁢                                                            {                                              1                        -                        τ                        +                                                  τⅇ                                                      j                            ⁢                                                          xe2x80x83                                                        ⁢                            T                            ⁢                                                          xe2x80x83                                                        ⁢                                                          (                                                              ω                                -                                                                  k                                  ⁢                                                                                                            2                                      ⁢                                      π                                                                        T                                                                                                                              )                                                                                                                          }                                        ·                                          F                      ⁡                                              (                                                  ω                          -                                                      k                            ⁢                                                                                          2                                ⁢                                π                                                            T                                                                                                      )                                                              ·                                                                                                                                          xe2x80x83                            ⁢                                                ⅇ                                      j                    ⁢                                          xe2x80x83                                        ⁢                    τ                    ⁢                                          xe2x80x83                                        ⁢                    T                    ⁢                                          xe2x80x83                                        ⁢                                          (                                              ω                        -                                                  k                          ⁢                                                                                    2                              ⁢                              π                                                        T                                                                                              )                                                                      ·                                  ⅇ                                                            -                      j                                        ⁢                                          xe2x80x83                                        ⁢                    2                    ⁢                    π                    ⁢                                          xe2x80x83                                        ⁢                    k                    ⁢                                          xe2x80x83                                        ⁢                    τ                                                                                                                          =                              xe2x80x83                            ⁢                                                1                  T                                ⁢                                                                            ∑                      ∞                                                              k                      =                                              -                        ∞                                                                              ⁢                                                            {                                              1                        -                        τ                        +                                                  τⅇ                                                      j                            ⁢                                                          xe2x80x83                                                        ⁢                            T                            ⁢                                                          xe2x80x83                                                        ⁢                                                          (                                                              ω                                -                                                                  k                                  ⁢                                                                                                            2                                      ⁢                                      π                                                                        T                                                                                                                              )                                                                                                                          }                                        ·                                          F                      ⁡                                              (                                                  ω                          -                                                      k                            ⁢                                                                                          2                                ⁢                                π                                                            T                                                                                                      )                                                              ·                                          ⅇ                                                                        -                          j                                                ⁢                                                  xe2x80x83                                                ⁢                        τ                        ⁢                                                  xe2x80x83                                                ⁢                        T                        ⁢                                                  xe2x80x83                                                ⁢                        ω                                                                                                                                                    (        6        )            
The signal represented by formula (6) is still in the form of infinite repetition at cycles of 2xcfx80/T of the signal whose frequency band is limited to xe2x88x92xcfx80/T less than xcfx89 less than xcfx80/T. When extracting the frequency components in the range of xe2x88x92xcfx80/T less than xcfx89 less than xcfx80/T by an ideal low-pass filter, a signal Fsbxe2x80x2(xcfx89) represented by the following formula (7) is obtained, and a signal which is close to the original continuous signal in spectrum can be restored. However, unlike the signal represented by formula (4), the signal represented by formula (7) is a signal obtained by applying band-pass characteristics represented by filtering characteristics of the following formula (8) to the spectrum of the original signal.                                           Fsb            xe2x80x2                    ⁡                      (            ω            )                          =                              1            T                    ⁢                                    (                              1                -                τ                +                                  τⅇ                                      j                    ⁢                                          xe2x80x83                                        ⁢                    T                    ⁢                                          xe2x80x83                                        ⁢                    ω                                                              )                        ·                          ⅇ                                                -                  j                                ⁢                                  xe2x80x83                                ⁢                τ                ⁢                                  xe2x80x83                                ⁢                T                ⁢                                  xe2x80x83                                ⁢                ω                                      ·                          F              ⁡                              (                ω                )                                                                        (        7        )                                          filter          ⁡                      (                          ω              ,              τ                        )                          =                              1            T                    ⁢                                    (                              1                -                τ                +                                  τ                  ⁢                                      xe2x80x83                                    ⁢                                      ⅇ                                          j                      ⁢                                              xe2x80x83                                            ⁢                      T                      ⁢                                              xe2x80x83                                            ⁢                      ω                                                                                  )                        ·                          ⅇ                                                -                  j                                ⁢                                  xe2x80x83                                ⁢                τ                ⁢                                  xe2x80x83                                ⁢                T                ⁢                                  xe2x80x83                                ⁢                ω                                                                        (        8        )            
The gain characteristics of the filter represented by formula (8) are as shown by the following formula (9).                                                                         "LeftBracketingBar"                                  filter                  ⁡                                      (                                          ω                      ,                      τ                                        )                                                  "RightBracketingBar"                            =                                                1                  T                                ⁢                                  "LeftBracketingBar"                                      1                    -                    τ                    +                                          τ                      ⁢                                              xe2x80x83                                            ⁢                                              ⅇ                                                  j                          ⁢                                                      xe2x80x83                                                    ⁢                          T                          ⁢                                                      xe2x80x83                                                    ⁢                          ω                                                                                                      "RightBracketingBar"                                                                                                        =                                                1                  T                                ⁢                                                      {                                                                                            (                                                      1                            -                            τ                                                    )                                                2                                            +                                              τ                        2                                            +                                              2                        ⁢                                                  τ                          ⁡                                                      (                                                          1                              -                              τ                                                        )                                                                          ⁢                        cos                        ⁢                                                  xe2x80x83                                                ⁢                        T                        ⁢                                                  xe2x80x83                                                ⁢                        ω                                                              }                                                        1/2                                                                                                          (        9        )            
The formula in { } on the right side of formula (9) can be changed to the following formula (9a). Since 2(1xe2x88x92cos Txcfx89)xe2x89xa70, formula (9) is minimized when xcfx84=xc2xd and maximized when xcfx84=0 or 1. That is, attenuation is maximized when xcfx84=xc2xd, and is nullified when xcfx84=0 or 1.                                                         (                              1                -                τ                            )                        2                    +                      τ            2                    +                      2            ⁢                          τ              ⁡                              (                                  1                  -                  τ                                )                                      ⁢            cos            ⁢                          xe2x80x83                        ⁢            T            ⁢                          xe2x80x83                        ⁢            ω                          =                              2            ⁢                          (                              1                -                                  cos                  ⁢                                      xe2x80x83                                    ⁢                  T                  ⁢                                      xe2x80x83                                    ⁢                  ω                                            )                        ⁢                                          (                                  τ                  -                                      1/2                                                  )                            2                                -                                    1              2                        ⁢                          (                              1                -                                  cos                  ⁢                                      xe2x80x83                                    ⁢                  T                  ⁢                                      xe2x80x83                                    ⁢                  ω                                            )                                +          1                                    (                  9          ⁢          a                )            
Further, since                                                         (                              1                -                τ                            )                        2                    +                      τ            2                    +                      2            ⁢                          τ              ⁡                              (                                  1                  -                  τ                                )                                      ⁢            cos            ⁢                          xe2x80x83                        ⁢            T            ⁢                          xe2x80x83                        ⁢            ω                          =                              2            ⁢                          (                              1                -                                  cos                  ⁢                                      xe2x80x83                                    ⁢                  T                  ⁢                                      xe2x80x83                                    ⁢                  ω                                            )                        ⁢                          {                                                                    (                                          τ                      -                                              1/2                                                              )                                    2                                -                                  1                  4                                            }                                +          1                                    (                  9          ⁢          b                )            
and 0xe2x89xa6xcfx84xe2x89xa61, the characteristics of amplitude factor when xcfx84 is fixed are as shown in FIG. 9. Further, it will be understood from FIG. 9 that the filter represented by formula (8) has low-pass characteristics and attenuation of high-frequency components depends on the value of xcfx84. The characteristics of the filter when xcfx84=xc2xd are as represented by the following formula (10).                               "LeftBracketingBar"                      filter            ⁡                          (                              ω                ,                                  1                  2                                            )                                "RightBracketingBar"                =                              1            T                    ⁢                      "LeftBracketingBar"                          cos              ⁢                              xe2x80x83                            ⁢                              T                2                            ⁢              ω                        "RightBracketingBar"                                              (        10        )            
Analysis of low-pass characteristics generated by linear interpolation of a one-dimensional signal is carried out in the manner described above. The similar characteristics are generated when interpolation is carried out on a two-dimensional sampled signal such as an image signal. Low-pass characteristics generated when a sampled signal obtained by translation of a two-dimensional sampled signal is obtained by interpolation approximation of the original two-dimensional signal are as represented by the following formula (11), wherein Tx and Ty respectively represent the sampling cycles in the directions of x-axis and y-axis, and accordingly, (dx, dy) represents the amount of translation of the sampling positions normalized by the sampling cycles.                               "LeftBracketingBar"                      filter            ⁡                          (                              u                ,                v                ,                dx                ,                dy                            )                                "RightBracketingBar"                =                              1                                          T                x                            ⁢                              T                y                                              ⁢                                    "LeftBracketingBar"                              1                -                dx                +                                  dx                  ·                                      ⅇ                                          j                      ⁢                                              xe2x80x83                                            ⁢                                              T                        x                                            ⁢                      u                                                                                  "RightBracketingBar"                        ·                          "LeftBracketingBar"                              1                -                dy                +                                  dy                  ·                                      ⅇ                                          j                      ⁢                                              xe2x80x83                                            ⁢                                              T                        y                                            ⁢                      v                                                                                  "RightBracketingBar"                                                          (        11        )            
The low-pass characteristics represented by formula (11) are generated when sampled values in the interpolating positions are obtained from values of original four sampling positions surrounding each interpolating positions by linear interpolation approximation represented by the following formula (12).
pi(x+dxTx, y+dyTy)=(1xe2x88x92dx)(1xe2x88x92dy)xc2x7p(x, y)+dx(1xe2x88x92dy)xc2x7p(x+Tx, y)+(1xe2x88x92dx)dyxc2x7p(x, y+Ty)+dxxc2x7dyxc2x7p(x+Tx, y+Ty)xe2x80x83xe2x80x83(12)
As in the one-dimensional signal described above, also the filter represented by formula (11) has low-pass characteristics where attenuation of high-frequency components changes depending upon the normalized position deviation between the original sampling positions and the interpolating sampling positions. The attenuation of the high-frequency components is minimized (or nullified) when dx=0 or 1 and dy=0 or 1, and maximized when dx=xc2xd and dy=xc2xd. The characteristics of the filter when dx=xc2xd and dy=xc2xd are as represented by the following formula (13).                               "LeftBracketingBar"                      filter            ⁡                          (                              u                ,                v                ,                                  1                  2                                ,                                  1                  2                                            )                                "RightBracketingBar"                =                              1                                          T                x                            ⁢                              T                y                                              ⁢                      "LeftBracketingBar"                          cos              ⁢                              xe2x80x83                            ⁢                                                                                          T                      x                                        ⁢                    u                                    2                                ·                cos                            ⁢                                                                    T                    y                                    ⁢                  v                                2                                      "RightBracketingBar"                                              (        13        )            
As can be understood from the description above, when linear interpolation is. carried out, there is generated deterioration of high-frequency components which depends upon the interpolating positions. Accordingly, there arises a problem that when retarding of a sound signal or translation of an image signal is carried out by linear interpolation processing, the frequency characteristics of the processed signal become different from those of the original signal due to difference in the amount of retardation or the amount of translation.
Further, when fine adjustment of an animation image signal, rotation of an image by a fine angle, fine adjustment of the rate of enlarging or contracting or fine adjustment of sound pitches is effected by linear interpolation, a more complicated problem arises. When the amount of adjustment is very small, the deviations between sampling positions and interpolating positions are substantially uniform so long as the signal is locally seen. However when the signal is seen in perspective, the position deviations largely fluctuate periodically and the image periodically becomes unsharp or sound periodically gets cracked.
In view of the foregoing observations and description, the primary object of the present invention is to provide a signal processing method and a signal processing system which can prevent deterioration of high-frequency components or generation of unsharpness in an image when linear interpolation processing is carried out.
In accordance with a first aspect of the present invention, there is provided a signal processing method for carrying out, on a discrete signal made up of signal components obtained by sampling an original signal in a plurality of sampling positions, linear interpolation processing to obtain interpolation signal components for interpolating positions other than the sampling positions, wherein the improvement comprises the step of
carrying out, on the linearly interpolated signal components obtained by carrying out the linear interpolation processing on the discrete signal, variable characteristic filtering processing for compensating for low-pass characteristics in the linear interpolation processing according to the position deviations between the sampling positions and the interpolating positions.
In the signal processing method of this invention, it is preferred that the variable characteristic filtering processing be carried out by use of a filter having as a transfer function an inverse function of the low-pass characteristics in the linear interpolation processing.
In the signal processing method of this invention, it is also preferred that the variable characteristic filtering processing be filtering processing in which the strength of application of a differential operator is changed according to the position deviations between the sampling positions and the interpolating positions.
In accordance with a second aspect of the present invention, there is provided a signal processing system for carrying out, on a discrete signal made up of signal components obtained by sampling an original signal in a plurality of sampling positions, linear interpolation processing to obtain interpolation signal components for interpolating positions other than the sampling positions, wherein the improvement comprises
a filtering processor means which carries out, on the linearly interpolated signal components obtained by carrying out the linear interpolation processing on the discrete signal, variable characteristic filtering processing for compensating for low-pass characteristics in the linear interpolation processing according to the position deviations between the sampling positions and the interpolating positions.
In the signal processing system of this invention, it is preferred that the filtering processor means carries out the variable characteristic filtering processing by use of a filter having as a transfer function an inverse function of the low-pass characteristics in the linear interpolation processing, or by changing the strength of application of a differential operator according to the position deviations between the sampling positions and the interpolating positions.
In the signal processing method and the signal processing system of the present invention, since variable characteristic filtering processing in which low-pass characteristics in the linear interpolation processing are compensated for according to the position deviations between the sampling positions and the interpolating positions is carried out on the interpolation signal components, deterioration in high frequency components of a signal, which gives rise to a problem especially in a digital signal processing involving linear interpolation, can be effectively corrected. Further by applying the present invention to image processing or sound signal processing, the phenomenon that the image periodically becomes unsharp when fine adjustment of an animation image signal, rotation of an image by a fine angle, or fine adjustment of the rate of enlarging or contracting is effected or the phenomenon that sound periodically gets cracked when fine adjustment of sound pitches is effected can be suppressed.